2024
Journal article
Open Access
Training a shallow NN to erase ink seepage in historical manuscripts based on a degradation model
Savino P., Tonazzini A.
In historical recto-verso manuscripts, very often the text written on the opposite page of the folio penetrates through the fiber of the paper, so that the texts on the two sides appear mixed. This is a very impairing damage that cannot be physically removed, and hinders both the work of philologists and palaeographers and the automatic analysis of linguistic contents. A procedure based on neural networks (NN) is proposed here to clean up the complex background of the manuscripts from this interference. We adopt a very simple shallow NN whose learning phase employs a training set generated from the data itself using a theoretical blending model that takes into account ink diffusion and saturation. By virtue of the parametric nature of the model, various levels of damage can be simulated in the training set, favoring a generalization capability of the NN. More explicitly, the network can be trained without the need for a large class of other similar manuscripts, but is still able, at least to some extent, to classify manuscripts with varying degrees of corruption. We compare the performance of this NN and other methods both qualitatively and quantitatively on a reference dataset and heavily damaged historical manuscripts.Source: Neural computing & applications (Print) (2024). doi:10.1007/s00521-023-09354-7
DOI: 10.1007/s00521-023-09354-7
Metrics:
Savino P., Tonazzini A.
In historical recto-verso manuscripts, very often the text written on the opposite page of the folio penetrates through the fiber of the paper, so that the texts on the two sides appear mixed. This is a very impairing damage that cannot be physically removed, and hinders both the work of philologists and palaeographers and the automatic analysis of linguistic contents. A procedure based on neural networks (NN) is proposed here to clean up the complex background of the manuscripts from this interference. We adopt a very simple shallow NN whose learning phase employs a training set generated from the data itself using a theoretical blending model that takes into account ink diffusion and saturation. By virtue of the parametric nature of the model, various levels of damage can be simulated in the training set, favoring a generalization capability of the NN. More explicitly, the network can be trained without the need for a large class of other similar manuscripts, but is still able, at least to some extent, to classify manuscripts with varying degrees of corruption. We compare the performance of this NN and other methods both qualitatively and quantitatively on a reference dataset and heavily damaged historical manuscripts.Source: Neural computing & applications (Print) (2024). doi:10.1007/s00521-023-09354-7
DOI: 10.1007/s00521-023-09354-7
Metrics:
See at:
ISTI Repository 
| link.springer.com 
| CNR ExploRA
2023
Conference article
Open Access
Mathematical models and neural networks for the description and the correction of typical distortions of historical manuscripts
Savino P., Tonazzini A.
Historical manuscripts are very often degraded by the seeping or transparency of the ink from the page opposite side. Suppressing the interfering text can be of great aid to philologists and paleographers who aim at interpreting the primary text, and nowadays also for the automatic analysis of the text. We formerly proposed a data model, which approximately describes this damage, to generate an artificial training set able to teach a shallow neural network how to classify pixels in clean or corrupted. This NN has proved to be effective in classifying manuscripts where the degradation can be also widely variable. In this paper, we modify the architecture of the NN to better account for ink saturation in text overlay areas, by including a specific class for these pixels. From the experiments, the improvement of the classification and then the restoration is significant.Source: ICCSA 2023 Workshops, pp. 545–557, Athens, Greece, 3-6/07/2023
DOI: 10.1007/978-3-031-37117-2_37
Metrics:
Savino P., Tonazzini A.
Historical manuscripts are very often degraded by the seeping or transparency of the ink from the page opposite side. Suppressing the interfering text can be of great aid to philologists and paleographers who aim at interpreting the primary text, and nowadays also for the automatic analysis of the text. We formerly proposed a data model, which approximately describes this damage, to generate an artificial training set able to teach a shallow neural network how to classify pixels in clean or corrupted. This NN has proved to be effective in classifying manuscripts where the degradation can be also widely variable. In this paper, we modify the architecture of the NN to better account for ink saturation in text overlay areas, by including a specific class for these pixels. From the experiments, the improvement of the classification and then the restoration is significant.Source: ICCSA 2023 Workshops, pp. 545–557, Athens, Greece, 3-6/07/2023
DOI: 10.1007/978-3-031-37117-2_37
Metrics:
See at:
ISTI Repository | link.springer.com | CNR ExploRA
2023
Conference article
Unknown
Preprocessing of recto-verso printed documents based on neural networks for text analysis
Savino P., Tonazzini A.
Among the many and varied damages affecting ancient documents, the penetration of ink from one side of the page to the other is one of the most frequent and invasive. In this work, we are interested in binarizing such degraded documents, for the application of OCR or other automatic text analysis tools, which can help philologists and palaeographers in text transcription. We previously proposed a data model that roughly describes this damage for front-to-back documents, and used it to generate an artificial training set that can teach a shallow neural network how to classify pixels on both sides into clean or corrupt. We show that this joint processing of the two sides of the document can significantly improve binarization and therefore OCR and other text analysis tasks, compared to the separate processing of the single sides, using the same information.Source: 3rd Conference on Digital Preservation and processing technology of Written Heritage, in conjunction with the 7th IEEE International Congress on Information Science and Technology (IEEE CiSt'23), Agadir - Essaouira, Morocco, 16-22/12/2023
Savino P., Tonazzini A.
Among the many and varied damages affecting ancient documents, the penetration of ink from one side of the page to the other is one of the most frequent and invasive. In this work, we are interested in binarizing such degraded documents, for the application of OCR or other automatic text analysis tools, which can help philologists and palaeographers in text transcription. We previously proposed a data model that roughly describes this damage for front-to-back documents, and used it to generate an artificial training set that can teach a shallow neural network how to classify pixels on both sides into clean or corrupt. We show that this joint processing of the two sides of the document can significantly improve binarization and therefore OCR and other text analysis tasks, compared to the separate processing of the single sides, using the same information.Source: 3rd Conference on Digital Preservation and processing technology of Written Heritage, in conjunction with the 7th IEEE International Congress on Information Science and Technology (IEEE CiSt'23), Agadir - Essaouira, Morocco, 16-22/12/2023
See at: CNR ExploRA
2023
Report
Open Access
AIMH Research Activities 2023
Aloia N., Amato G., Bartalesi V., Bianchi L., Bolettieri P., Bosio C., Carraglia M., Carrara F., Casarosa V., Ciampi L., Coccomini D. A., Concordia C., Corbara S., De Martino C., Di Benedetto M., Esuli A., Falchi F., Fazzari E., Gennaro C., Lagani G., Lenzi E., Meghini C., Messina N., Molinari A., Moreo A., Nardi A., Pedrotti A., Pratelli N., Puccetti G., Rabitti F., Savino P., Sebastiani F., Sperduti G., Thanos C., Trupiano L., Vadicamo L., Vairo C., Versienti L.
The AIMH (Artificial Intelligence for Media and Humanities) laboratory is dedicated to exploring and pushing the boundaries in the field of Artificial Intelligence, with a particular focus on its application in digital media and humanities. This lab's objective is to enhance the current state of AI technology particularly on deep learning, text analysis, computer vision, multimedia information retrieval, multimedia content analysis, recognition, and retrieval. This report encapsulates the laboratory's progress and activities throughout the year 2023.Source: ISTI Annual Reports, 2023
DOI: 10.32079/isti-ar-2023/001
Metrics:
Aloia N., Amato G., Bartalesi V., Bianchi L., Bolettieri P., Bosio C., Carraglia M., Carrara F., Casarosa V., Ciampi L., Coccomini D. A., Concordia C., Corbara S., De Martino C., Di Benedetto M., Esuli A., Falchi F., Fazzari E., Gennaro C., Lagani G., Lenzi E., Meghini C., Messina N., Molinari A., Moreo A., Nardi A., Pedrotti A., Pratelli N., Puccetti G., Rabitti F., Savino P., Sebastiani F., Sperduti G., Thanos C., Trupiano L., Vadicamo L., Vairo C., Versienti L.
The AIMH (Artificial Intelligence for Media and Humanities) laboratory is dedicated to exploring and pushing the boundaries in the field of Artificial Intelligence, with a particular focus on its application in digital media and humanities. This lab's objective is to enhance the current state of AI technology particularly on deep learning, text analysis, computer vision, multimedia information retrieval, multimedia content analysis, recognition, and retrieval. This report encapsulates the laboratory's progress and activities throughout the year 2023.Source: ISTI Annual Reports, 2023
DOI: 10.32079/isti-ar-2023/001
Metrics:
See at:
ISTI Repository | CNR ExploRA
2022
Journal article
Open Access
Blind bleed-through removal in color ancient manuscripts
Hanif M., Tonazzini A., Hussain S. F., Habib U., Salerno E., Savino P., Halim Z.
Archaic manuscripts are an important part of ancient civilization. Unfortunately, such documents are often affected by various age related degradations, which impinge their legibility and information contents, and destroy their original look. In general, these documents are composed of three layers of information: foreground text, background, and unwanted degradation in the form of patterns interfering with the main text. In this work, we are presenting a color space based image segmentation technique to separate and remove the bleed-through degradation in digital ancient manuscripts. The main theme is to improve their readability and restore their original aesthetic look. For each pixel, a feature vector is created using color spectral and spatial location information. A pixel based segmentation method using Gaussian Mixture Model (GMM) is employed, assuming that each feature vector corresponds to a Gaussian distribution. Based on this assumption, each pixel is supposed to be drawn from a mixture of Gaussian distribution, with unknown parameters. The Expectation-Maximization (EM) approach is then used to estimate the unknown GMM parameters. The appropriate class label for each pixel is then estimated using posterior probability and GMM parameters. Unlike other binarization based document restoration method where the focus is on text extraction, we are more interested in restoring the aesthetically pleasing look of the ancient documents.The experimental results validate the usefulness of proposed method in terms of successful bleed-through identification and removal, while preserving foreground-text and background information.Source: Multimedia tools and applications (Dordrecht. Online) 82 (2022): 12321–12335. doi:10.1007/s11042-022-13755-6
DOI: 10.1007/s11042-022-13755-6
Metrics:
Hanif M., Tonazzini A., Hussain S. F., Habib U., Salerno E., Savino P., Halim Z.
Archaic manuscripts are an important part of ancient civilization. Unfortunately, such documents are often affected by various age related degradations, which impinge their legibility and information contents, and destroy their original look. In general, these documents are composed of three layers of information: foreground text, background, and unwanted degradation in the form of patterns interfering with the main text. In this work, we are presenting a color space based image segmentation technique to separate and remove the bleed-through degradation in digital ancient manuscripts. The main theme is to improve their readability and restore their original aesthetic look. For each pixel, a feature vector is created using color spectral and spatial location information. A pixel based segmentation method using Gaussian Mixture Model (GMM) is employed, assuming that each feature vector corresponds to a Gaussian distribution. Based on this assumption, each pixel is supposed to be drawn from a mixture of Gaussian distribution, with unknown parameters. The Expectation-Maximization (EM) approach is then used to estimate the unknown GMM parameters. The appropriate class label for each pixel is then estimated using posterior probability and GMM parameters. Unlike other binarization based document restoration method where the focus is on text extraction, we are more interested in restoring the aesthetically pleasing look of the ancient documents.The experimental results validate the usefulness of proposed method in terms of successful bleed-through identification and removal, while preserving foreground-text and background information.Source: Multimedia tools and applications (Dordrecht. Online) 82 (2022): 12321–12335. doi:10.1007/s11042-022-13755-6
DOI: 10.1007/s11042-022-13755-6
Metrics:
See at:
ISTI Repository | CNR ExploRA
2022
Conference article
Open Access
A shallow neural net with model-based learning for the virtual restoration of recto-verso manuscript
Savino P., Tonazzini A.
We propose a fast procedure based on neural networks (NN) to correct the typically complex background of recto-verso historical manuscripts, where the texts of the two sides often appear mixed. The purpose is to eliminate the interfering, shining-through text, to facilitate both the work of philologists and paleographers and the automatic analysis of the linguistic contents. We adapt the learning phase of a very simple shallow NN to exploit the information of the registered recto and verso sides of the manuscript without the need for a large class of other similar manuscripts. Hence, the training set is self-generated from the data images based on a theoretical mixing model that accounts for ink spreading through the paper fiber and for ink saturation in the text superposition areas. Operationally, we select pairs of patches containing clean text from the manuscript and then mix them symmetrically using the model with varying parameters that span the allowed range. This makes the NN able to generalize to diverse amounts of ink seeping and then classify different manuscripts. We show comparisons between the results obtained on heavily damaged manuscripts with this NN and other approaches. From a qualitative point of view, the proposed method seems quite promising.Source: VIPERC2022: 1st International Virtual Conference on Visual Pattern Extraction and Recognition for Cultural Heritage Understanding, Online event, 12/09/2022
Savino P., Tonazzini A.
We propose a fast procedure based on neural networks (NN) to correct the typically complex background of recto-verso historical manuscripts, where the texts of the two sides often appear mixed. The purpose is to eliminate the interfering, shining-through text, to facilitate both the work of philologists and paleographers and the automatic analysis of the linguistic contents. We adapt the learning phase of a very simple shallow NN to exploit the information of the registered recto and verso sides of the manuscript without the need for a large class of other similar manuscripts. Hence, the training set is self-generated from the data images based on a theoretical mixing model that accounts for ink spreading through the paper fiber and for ink saturation in the text superposition areas. Operationally, we select pairs of patches containing clean text from the manuscript and then mix them symmetrically using the model with varying parameters that span the allowed range. This makes the NN able to generalize to diverse amounts of ink seeping and then classify different manuscripts. We show comparisons between the results obtained on heavily damaged manuscripts with this NN and other approaches. From a qualitative point of view, the proposed method seems quite promising.Source: VIPERC2022: 1st International Virtual Conference on Visual Pattern Extraction and Recognition for Cultural Heritage Understanding, Online event, 12/09/2022
See at:
ceur-ws.org | ISTI Repository | ISTI Repository | CNR ExploRA
2022
Report
Open Access
AIMH research activities 2022
Aloia N., Amato G., Bartalesi V., Benedetti F., Bolettieri P., Cafarelli D., Carrara F., Casarosa V., Ciampi L., Coccomini D. A., Concordia C., Corbara S., Di Benedetto M., Esuli A., Falchi F., Gennaro C., Lagani G., Lenzi E., Meghini C., Messina N., Metilli D., Molinari A., Moreo A., Nardi A., Pedrotti A., Pratelli N., Rabitti F., Savino P., Sebastiani F., Sperduti G., Thanos C., Trupiano L., Vadicamo L., Vairo C.
The Artificial Intelligence for Media and Humanities laboratory (AIMH) has the mission to investigate and advance the state of the art in the Artificial Intelligence field, specifically addressing applications to digital media and digital humanities, and taking also into account issues related to scalability. This report summarize the 2022 activities of the research group.Source: ISTI Annual reports, 2022
DOI: 10.32079/isti-ar-2022/002
Metrics:
Aloia N., Amato G., Bartalesi V., Benedetti F., Bolettieri P., Cafarelli D., Carrara F., Casarosa V., Ciampi L., Coccomini D. A., Concordia C., Corbara S., Di Benedetto M., Esuli A., Falchi F., Gennaro C., Lagani G., Lenzi E., Meghini C., Messina N., Metilli D., Molinari A., Moreo A., Nardi A., Pedrotti A., Pratelli N., Rabitti F., Savino P., Sebastiani F., Sperduti G., Thanos C., Trupiano L., Vadicamo L., Vairo C.
The Artificial Intelligence for Media and Humanities laboratory (AIMH) has the mission to investigate and advance the state of the art in the Artificial Intelligence field, specifically addressing applications to digital media and digital humanities, and taking also into account issues related to scalability. This report summarize the 2022 activities of the research group.Source: ISTI Annual reports, 2022
DOI: 10.32079/isti-ar-2022/002
Metrics:
See at:
ISTI Repository | CNR ExploRA
2021
Journal article
Open Access
A procedure for the correction of back-to-front degradations in archival manuscripts with preservation of the original appearance
Savino P., Tonazzini A.
Virtual restoration of digital copies of the human documental heritage is crucial for facilitating both the traditional work of philologists and paleographers and the automatic analysis of the contents. Here we propose a practical and fast procedure for the correction of the typically complex background of recto-verso historical manuscripts. The procedure has two main, distinctive features: it does not need for a preliminary registration of the two page sides, and it is non-invasive, as it does not alter the original appearance of the manuscript. This makes it suitable for the routinary use in the archives, and permits an easier fruition of the manuscripts, without any information being lost. In the ¯rst stage, the detection of both the primary text and the spurious strokes is performed via soft segmentation, based on the statistical decorrelation of the two recto and verso images. In the second stage, the noisy pattern is substituted with pixels that simulate the texture of the clean surrounding background, through an e±cient technique of image inpainting. As shown in the experimental results, evaluated both qualitatively and quantitatively, the proposed procedure is able to perform a ¯ne and selective removal of the degradation, while preserving other informative marks of the manuscript history.Source: Vietnam journal of computer science (Online) (2021). doi:10.1142/S2196888822500099
DOI: 10.1142/s2196888822500099
Metrics:
Savino P., Tonazzini A.
Virtual restoration of digital copies of the human documental heritage is crucial for facilitating both the traditional work of philologists and paleographers and the automatic analysis of the contents. Here we propose a practical and fast procedure for the correction of the typically complex background of recto-verso historical manuscripts. The procedure has two main, distinctive features: it does not need for a preliminary registration of the two page sides, and it is non-invasive, as it does not alter the original appearance of the manuscript. This makes it suitable for the routinary use in the archives, and permits an easier fruition of the manuscripts, without any information being lost. In the ¯rst stage, the detection of both the primary text and the spurious strokes is performed via soft segmentation, based on the statistical decorrelation of the two recto and verso images. In the second stage, the noisy pattern is substituted with pixels that simulate the texture of the clean surrounding background, through an e±cient technique of image inpainting. As shown in the experimental results, evaluated both qualitatively and quantitatively, the proposed procedure is able to perform a ¯ne and selective removal of the degradation, while preserving other informative marks of the manuscript history.Source: Vietnam journal of computer science (Online) (2021). doi:10.1142/S2196888822500099
DOI: 10.1142/s2196888822500099
Metrics:
See at:
ISTI Repository | ISTI Repository | www.worldscientific.com | Vietnam Journal of Computer Science | CNR ExploRA
2021
Report
Open Access
AIMH research activities 2021
Aloia N., Amato G., Bartalesi V., Benedetti F., Bolettieri P., Cafarelli D., Carrara F., Casarosa V., Coccomini D., Ciampi L., Concordia C., Corbara S., Di Benedetto M., Esuli A., Falchi F., Gennaro C., Lagani G., Massoli F. V., Meghini C., Messina N., Metilli D., Molinari A., Moreo A., Nardi A., Pedrotti A., Pratelli N., Rabitti F., Savino P., Sebastiani F., Sperduti G., Thanos C., Trupiano L., Vadicamo L., Vairo C.
The Artificial Intelligence for Media and Humanities laboratory (AIMH) has the mission to investigate and advance the state of the art in the Artificial Intelligence field, specifically addressing applications to digital media and digital humanities, and taking also into account issues related to scalability. This report summarize the 2021 activities of the research group.Source: ISTI Annual Report, ISTI-2021-AR/003, pp.1–34, 2021
DOI: 10.32079/isti-ar-2021/003
Metrics:
Aloia N., Amato G., Bartalesi V., Benedetti F., Bolettieri P., Cafarelli D., Carrara F., Casarosa V., Coccomini D., Ciampi L., Concordia C., Corbara S., Di Benedetto M., Esuli A., Falchi F., Gennaro C., Lagani G., Massoli F. V., Meghini C., Messina N., Metilli D., Molinari A., Moreo A., Nardi A., Pedrotti A., Pratelli N., Rabitti F., Savino P., Sebastiani F., Sperduti G., Thanos C., Trupiano L., Vadicamo L., Vairo C.
The Artificial Intelligence for Media and Humanities laboratory (AIMH) has the mission to investigate and advance the state of the art in the Artificial Intelligence field, specifically addressing applications to digital media and digital humanities, and taking also into account issues related to scalability. This report summarize the 2021 activities of the research group.Source: ISTI Annual Report, ISTI-2021-AR/003, pp.1–34, 2021
DOI: 10.32079/isti-ar-2021/003
Metrics:
See at:
ISTI Repository | CNR ExploRA
2020
Report
Open Access
AIMH research activities 2020
Aloia N., Amato G., Bartalesi V., Benedetti F., Bolettieri P., Carrara F., Casarosa V., Ciampi L., Concordia C., Corbara S., Esuli A., Falchi F., Gennaro C., Lagani G., Massoli F. V., Meghini C., Messina N., Metilli D., Molinari A., Moreo A., Nardi A., Pedrotti A., Pratelli N., Rabitti F., Savino P., Sebastiani F., Thanos C., Trupiano L., Vadicamo L., Vairo C.
Annual Report of the Artificial Intelligence for Media and Humanities laboratory (AIMH) research activities in 2020.Source: ISTI Annual Report, ISTI-2020-AR/001, 2020
DOI: 10.32079/isti-ar-2020/001
Metrics:
Aloia N., Amato G., Bartalesi V., Benedetti F., Bolettieri P., Carrara F., Casarosa V., Ciampi L., Concordia C., Corbara S., Esuli A., Falchi F., Gennaro C., Lagani G., Massoli F. V., Meghini C., Messina N., Metilli D., Molinari A., Moreo A., Nardi A., Pedrotti A., Pratelli N., Rabitti F., Savino P., Sebastiani F., Thanos C., Trupiano L., Vadicamo L., Vairo C.
Annual Report of the Artificial Intelligence for Media and Humanities laboratory (AIMH) research activities in 2020.Source: ISTI Annual Report, ISTI-2020-AR/001, 2020
DOI: 10.32079/isti-ar-2020/001
Metrics:
See at:
ISTI Repository | CNR ExploRA
2019
Journal article
Open Access
Virtual restoration and content analysis of ancient degraded manuscripts
Tonazzini A., Savino P., Salerno E., Hanif M., Debole F.
In recent years, extensive campaigns of digitization of the documental heritage conserved in libraries and archives have been performed, with the primary goal to ensure the preservation and fruition of this important part of the human cultural and historical patrimony. Besides protecting conservation, the availability of high quality digital copies has increasingly stimulated the use of image processing techniques, to perform a number of operations on documents and manuscripts, without harming the often precious and fragile originals. Among those, virtual restoration tasks are crucial, as they facilitate the traditional work of philologists and paleographers, and constitute a first step towards an automatic analysis of the written contents. Here we report our experience in this field, referring, as a case study, to the problem of removing one of the most frequent and impairing degradations affecting ancient manuscripts, i.e., the bleed-through distortion.We show that techniques of blind source separation are versatile tools to either cancel these unwanted interferences or isolate specific features for content analysis goals. Specialized algorithms, based on recto-verso models and sparse image representation, are then shown to be able to perform a fine and selective removal of the degradation, while preserving the original appearance of the manuscript.Source: International Journal of Information Science and Technology 3 (2019): 16–25. doi:10.57675/IMIST.PRSM/ijist-v3i5.133
DOI: 10.57675/imist.prsm/ijist-v3i5.133
Metrics:
Tonazzini A., Savino P., Salerno E., Hanif M., Debole F.
In recent years, extensive campaigns of digitization of the documental heritage conserved in libraries and archives have been performed, with the primary goal to ensure the preservation and fruition of this important part of the human cultural and historical patrimony. Besides protecting conservation, the availability of high quality digital copies has increasingly stimulated the use of image processing techniques, to perform a number of operations on documents and manuscripts, without harming the often precious and fragile originals. Among those, virtual restoration tasks are crucial, as they facilitate the traditional work of philologists and paleographers, and constitute a first step towards an automatic analysis of the written contents. Here we report our experience in this field, referring, as a case study, to the problem of removing one of the most frequent and impairing degradations affecting ancient manuscripts, i.e., the bleed-through distortion.We show that techniques of blind source separation are versatile tools to either cancel these unwanted interferences or isolate specific features for content analysis goals. Specialized algorithms, based on recto-verso models and sparse image representation, are then shown to be able to perform a fine and selective removal of the degradation, while preserving the original appearance of the manuscript.Source: International Journal of Information Science and Technology 3 (2019): 16–25. doi:10.57675/IMIST.PRSM/ijist-v3i5.133
DOI: 10.57675/imist.prsm/ijist-v3i5.133
Metrics:
See at:
ISTI Repository | International Journal of Information Science and Technology | www.innove.org | doi.org | CNR ExploRA
2019
Report
Open Access
AIMIR 2019 Research Activities
Amato G., Bolettieri P., Carrara F., Ciampi L., Di Benedetto M., Debole F., Falchi F., Gennaro C., Lagani G., Massoli F. V., Messina N., Rabitti F., Savino P., Vadicamo L., Vairo C.
Multimedia Information Retrieval (AIMIR) research group is part of the NeMIS laboratory of the Information Science and Technologies Institute "A. Faedo" (ISTI) of the Italian National Research Council (CNR). The AIMIR group has a long experience in topics related to: Artificial Intelligence, Multimedia Information Retrieval, Computer Vision and Similarity search on a large scale. We aim at investigating the use of Artificial Intelligence and Deep Learning, for Multimedia Information Retrieval, addressing both effectiveness and efficiency. Multimedia information retrieval techniques should be able to provide users with pertinent results, fast, on huge amount of multimedia data. Application areas of our research results range from cultural heritage to smart tourism, from security to smart cities, from mobile visual search to augmented reality. This report summarize the 2019 activities of the research group.Source: AIMIR Annual Report, 2019
Amato G., Bolettieri P., Carrara F., Ciampi L., Di Benedetto M., Debole F., Falchi F., Gennaro C., Lagani G., Massoli F. V., Messina N., Rabitti F., Savino P., Vadicamo L., Vairo C.
Multimedia Information Retrieval (AIMIR) research group is part of the NeMIS laboratory of the Information Science and Technologies Institute "A. Faedo" (ISTI) of the Italian National Research Council (CNR). The AIMIR group has a long experience in topics related to: Artificial Intelligence, Multimedia Information Retrieval, Computer Vision and Similarity search on a large scale. We aim at investigating the use of Artificial Intelligence and Deep Learning, for Multimedia Information Retrieval, addressing both effectiveness and efficiency. Multimedia information retrieval techniques should be able to provide users with pertinent results, fast, on huge amount of multimedia data. Application areas of our research results range from cultural heritage to smart tourism, from security to smart cities, from mobile visual search to augmented reality. This report summarize the 2019 activities of the research group.Source: AIMIR Annual Report, 2019
See at:
ISTI Repository | CNR ExploRA
2019
Journal article
Open Access
Bleed-through cancellation in non-rigidly misaligned recto-verso archival manuscripts based on local registration
Savino P., Tonazzini A., Bedini L.
Ancient manuscripts written on both pages of the sheet are frequently affected by ink bleeding from the reverse side. This phenomenon produces a significant degradation of both the foreground text and the general appearance of the manuscript. Effective digital image restoration techniques may require the use of the content of both document sides, thus needing their perfect alignment. Although often available, recto and verso are usually not aligned, either for rigid misalignments occurring during acquisition, or for non-rigid deformations of the sheet. In this paper, we propose a novel method to restore color recto-verso manuscript images in a piecewise manner, without the need of a preliminary, global registration of the two sides. We assume that at the local level any deformation can be approximated by a displacement and subdivide the two images into small patches of same size. For each pair of patches at the same location, their relative shift is estimated by cross-correlation, thus allowing their straightforward alignment. A bleed-through removal algorithm is then applied to the registered patches. By spanning the entire images, this procedure returns free-of-interferences versions of the images in their original acquisition layout. The experiments show that the restoration results so obtained are better than those obtained with the classical approach that first registers the whole recto-verso pair and then performs restoration. Further advantages are a much lower computational cost, the possibility to manage non-global and non-rigid deformations, and the unaltered geometry and color appearance of the two restored images.Source: International journal on document analysis and recognition (Print) 22 (2019): 163–176. doi:10.1007/s10032-019-00323-2
DOI: 10.1007/s10032-019-00323-2
Metrics:
Savino P., Tonazzini A., Bedini L.
Ancient manuscripts written on both pages of the sheet are frequently affected by ink bleeding from the reverse side. This phenomenon produces a significant degradation of both the foreground text and the general appearance of the manuscript. Effective digital image restoration techniques may require the use of the content of both document sides, thus needing their perfect alignment. Although often available, recto and verso are usually not aligned, either for rigid misalignments occurring during acquisition, or for non-rigid deformations of the sheet. In this paper, we propose a novel method to restore color recto-verso manuscript images in a piecewise manner, without the need of a preliminary, global registration of the two sides. We assume that at the local level any deformation can be approximated by a displacement and subdivide the two images into small patches of same size. For each pair of patches at the same location, their relative shift is estimated by cross-correlation, thus allowing their straightforward alignment. A bleed-through removal algorithm is then applied to the registered patches. By spanning the entire images, this procedure returns free-of-interferences versions of the images in their original acquisition layout. The experiments show that the restoration results so obtained are better than those obtained with the classical approach that first registers the whole recto-verso pair and then performs restoration. Further advantages are a much lower computational cost, the possibility to manage non-global and non-rigid deformations, and the unaltered geometry and color appearance of the two restored images.Source: International journal on document analysis and recognition (Print) 22 (2019): 163–176. doi:10.1007/s10032-019-00323-2
DOI: 10.1007/s10032-019-00323-2
Metrics:
See at:
ISTI Repository | International Journal on Document Analysis and Recognition (IJDAR) | link.springer.com | CNR ExploRA
2019
Journal article
Open Access
A data model and a cataloguing, storage and retrieval system for ancient document archives
Savino P., Tonazzini A., Debole F.
Digitalization of ancient manuscripts is becoming a common practice in many archives and libraries, mainly for preservation purposes. This opens many new opportunities for the diffusion of these precious cultural assets, since several scholars and researchers, as well as the general public, may access and use them for research purposes, for study, and for general information. This is made possible if the documents, their descriptions, and the result of all processing activities performed on them are acquired at a good quality and can be easily accessed by using simple and powerful retrieval mechanisms. Acquired manuscripts suffer of degradations that may require different types of elaborations on the digital images, to improve their visual quality and legibility, or to discover hidden text that is not visible. Natural Language Processing requires the creation of transcriptions of the text contained in the manuscript, as well as encoding of the document structure and creation of user annotations. This paper presents a document management system and a metadata schema that make possible the storage and content-based retrieval of original documents, elaborations performed to improve their readability, textual transcriptions, and linguistic annotations. The archive will offer the possibility of describing, storing and accessing all the available manuscript versions, document transcriptions and annotations, and to search and retrieve documents based on all this information.Source: International Journal of Information Science and Technology 3 (2019): 6–15. doi:10.57675/IMIST.PRSM/ijist-v3i5.132
DOI: 10.57675/imist.prsm/ijist-v3i5.132
Metrics:
Savino P., Tonazzini A., Debole F.
Digitalization of ancient manuscripts is becoming a common practice in many archives and libraries, mainly for preservation purposes. This opens many new opportunities for the diffusion of these precious cultural assets, since several scholars and researchers, as well as the general public, may access and use them for research purposes, for study, and for general information. This is made possible if the documents, their descriptions, and the result of all processing activities performed on them are acquired at a good quality and can be easily accessed by using simple and powerful retrieval mechanisms. Acquired manuscripts suffer of degradations that may require different types of elaborations on the digital images, to improve their visual quality and legibility, or to discover hidden text that is not visible. Natural Language Processing requires the creation of transcriptions of the text contained in the manuscript, as well as encoding of the document structure and creation of user annotations. This paper presents a document management system and a metadata schema that make possible the storage and content-based retrieval of original documents, elaborations performed to improve their readability, textual transcriptions, and linguistic annotations. The archive will offer the possibility of describing, storing and accessing all the available manuscript versions, document transcriptions and annotations, and to search and retrieve documents based on all this information.Source: International Journal of Information Science and Technology 3 (2019): 6–15. doi:10.57675/IMIST.PRSM/ijist-v3i5.132
DOI: 10.57675/imist.prsm/ijist-v3i5.132
Metrics:
See at:
innove.org | ISTI Repository | International Journal of Information Science and Technology | doi.org | CNR ExploRA
2018
Journal article
Open Access
Sparse representation based inpainting for the restoration of document images affected by bleed-through
Hanif M., Tonazzini A., Savino P., Salerno E.
Bleed-through is a commonly encountered degradation in ancient printed documents and manuscripts, which severely impair their readability. Digital image restoration techniques can be effective to remove or significantly reduce this degradation. In bleed-through document image restoration the main issue is to identify the bleed-through pixels and replace them with appropriate values, in accordance to their surroundings. In this paper, we propose a two stage method, where a pair of properly registered images of the document recto and verso is first used to locate the bleed-through pixels in each side, and then a sparse representation based image inpainting technique is used to fill-in the bleed-through areas according to the neighbourhood, in such a way to preserve the original appearance of the document. The advantages of the proposed inpainting technique over state-of-the-art methods are illustrated by the improvement in the visual results.Source: Proceedings (MDPI) 2 (2018). doi:10.3390/proceedings2020093
DOI: 10.3390/proceedings2020093
Metrics:
Hanif M., Tonazzini A., Savino P., Salerno E.
Bleed-through is a commonly encountered degradation in ancient printed documents and manuscripts, which severely impair their readability. Digital image restoration techniques can be effective to remove or significantly reduce this degradation. In bleed-through document image restoration the main issue is to identify the bleed-through pixels and replace them with appropriate values, in accordance to their surroundings. In this paper, we propose a two stage method, where a pair of properly registered images of the document recto and verso is first used to locate the bleed-through pixels in each side, and then a sparse representation based image inpainting technique is used to fill-in the bleed-through areas according to the neighbourhood, in such a way to preserve the original appearance of the document. The advantages of the proposed inpainting technique over state-of-the-art methods are illustrated by the improvement in the visual results.Source: Proceedings (MDPI) 2 (2018). doi:10.3390/proceedings2020093
DOI: 10.3390/proceedings2020093
Metrics:
See at:
doi.org | ISTI Repository | www.mdpi.com | www.mdpi.com | CNR ExploRA
2018
Journal article
Open Access
Non-local sparse image inpainting for document bleed-through removal
Hanif M., Tonazzini A., Savino P., Salerno E.
Bleed-through is a frequent, pervasive degradation in ancient manuscripts, which is caused by ink seeped from the opposite side of the sheet. Bleed-through, appearing as an extra interfering text, hinders document readability and makes it difficult to decipher the information contents. Digital image restoration techniques have been successfully employed to remove or significantly reduce this distortion. This paper proposes a two-step restoration method for documents affected by bleed-through, exploiting information from the recto and verso images. First, the bleed-through pixels are identified, based on a non-stationary, linear model of the two texts overlapped in the recto-verso pair. In the second step, a dictionary learning-based sparse image inpainting technique, with non-local patch grouping, is used to reconstruct the bleed-through-contaminated image information. An overcomplete sparse dictionary is learned from the bleed-through-free image patches, which is then used to estimate a befitting fill-in for the identified bleed-through pixels. The non-local patch similarity is employed in the sparse reconstruction of each patch, to enforce the local similarity. Thanks to the intrinsic image sparsity and non-local patch similarity, the natural texture of the background is well reproduced in the bleed-through areas, and even a possible overestimation of the bleed through pixels is effectively corrected, so that the original appearance of the document is preserved. We evaluate the performance of the proposed method on the images of a popular database of ancient documents, and the results validate the performance of the proposed method compared to the state of the art.Source: JOURNAL OF IMAGING 4 (2018). doi:10.3390/jimaging4050068
DOI: 10.3390/jimaging4050068
Metrics:
Hanif M., Tonazzini A., Savino P., Salerno E.
Bleed-through is a frequent, pervasive degradation in ancient manuscripts, which is caused by ink seeped from the opposite side of the sheet. Bleed-through, appearing as an extra interfering text, hinders document readability and makes it difficult to decipher the information contents. Digital image restoration techniques have been successfully employed to remove or significantly reduce this distortion. This paper proposes a two-step restoration method for documents affected by bleed-through, exploiting information from the recto and verso images. First, the bleed-through pixels are identified, based on a non-stationary, linear model of the two texts overlapped in the recto-verso pair. In the second step, a dictionary learning-based sparse image inpainting technique, with non-local patch grouping, is used to reconstruct the bleed-through-contaminated image information. An overcomplete sparse dictionary is learned from the bleed-through-free image patches, which is then used to estimate a befitting fill-in for the identified bleed-through pixels. The non-local patch similarity is employed in the sparse reconstruction of each patch, to enforce the local similarity. Thanks to the intrinsic image sparsity and non-local patch similarity, the natural texture of the background is well reproduced in the bleed-through areas, and even a possible overestimation of the bleed through pixels is effectively corrected, so that the original appearance of the document is preserved. We evaluate the performance of the proposed method on the images of a popular database of ancient documents, and the results validate the performance of the proposed method compared to the state of the art.Source: JOURNAL OF IMAGING 4 (2018). doi:10.3390/jimaging4050068
DOI: 10.3390/jimaging4050068
Metrics:
See at:
Journal of Imaging | ISTI Repository | DOAJ-Articles | Journal of Imaging | CNR ExploRA
2018
Conference article
Open Access
Document bleed-through removal using sparse image inpainting
Hanif M., Tonazzini A., Savino P., Salerno E., Tsagkatakis G.
Bleed-through is a pervasive degradation in ancient documents, caused by the ink of the opposite side of the sheet that has seeped through the paper fiber, and appears as an extra, interfering text. Bleed-through severely impairs document readability and makes it difficult to decipher the contents. Digital image restoration techniques have been successfully employed to remove or significantly reduce this distortion. The main theme is to identify the bleedthrough pixels and estimate an appropriate replacement for them, in accordance to their surrounding. This paper proposes a two-step image restoration method, exploiting information from the recto and verso images. First, based on a non-stationary linear model of the two texts overlapped in the recto-verso pair, the bleed-through pixels are identified. In the second step, a sparse representation based image inpainting technique, with a non-negative sparsity constraint, is used to find an appropriate replacement for the bleedthough pixels. Thanks to the power of dictionary learning and sparse image reconstruction methods, the natural texture of the background is well reproduced in the bleed-through areas, and even a their possible overestimation is effectively corrected, so that the original appearance of the document is preserved. The experiments are conducted on the images of a popular database of ancient documents, and the results validate the performance of the proposed method compared to the state of the art.Source: DAS 2018 - 13th IAPR International Workshop on Document Analysis Systems, pp. 281–286, Vienna, Austria, 24-27 April 2018
DOI: 10.1109/das.2018.21
Metrics:
Hanif M., Tonazzini A., Savino P., Salerno E., Tsagkatakis G.
Bleed-through is a pervasive degradation in ancient documents, caused by the ink of the opposite side of the sheet that has seeped through the paper fiber, and appears as an extra, interfering text. Bleed-through severely impairs document readability and makes it difficult to decipher the contents. Digital image restoration techniques have been successfully employed to remove or significantly reduce this distortion. The main theme is to identify the bleedthrough pixels and estimate an appropriate replacement for them, in accordance to their surrounding. This paper proposes a two-step image restoration method, exploiting information from the recto and verso images. First, based on a non-stationary linear model of the two texts overlapped in the recto-verso pair, the bleed-through pixels are identified. In the second step, a sparse representation based image inpainting technique, with a non-negative sparsity constraint, is used to find an appropriate replacement for the bleedthough pixels. Thanks to the power of dictionary learning and sparse image reconstruction methods, the natural texture of the background is well reproduced in the bleed-through areas, and even a their possible overestimation is effectively corrected, so that the original appearance of the document is preserved. The experiments are conducted on the images of a popular database of ancient documents, and the results validate the performance of the proposed method compared to the state of the art.Source: DAS 2018 - 13th IAPR International Workshop on Document Analysis Systems, pp. 281–286, Vienna, Austria, 24-27 April 2018
DOI: 10.1109/das.2018.21
Metrics:
See at:
ISTI Repository | doi.org | ieeexplore.ieee.org | CNR ExploRA
2018
Conference article
Open Access
Archiving and retrieving digital elaborations of ancient manuscripts
Savino P., Tonazzini A., Debole F., Salerno E.
Digitalization of ancient manuscripts is becoming a standard in libraries and archives. In many cases, manuscripts suffer of degradations that may require performing different types of elaborations on the digital images, in order to improve their legibility and analyze their contents. Digital archives containing digital images of manuscripts and all the elaborations performed on these images are thus of primary importance for a complete exploitation of all available information regarding the manuscripts themselves. This paper presents a metadata schema suitable for the management of such an archive. The archive will offer the possibility of describing, storing and accessing all the available manuscript versions, and to search them based on their content.Source: CiST 2018 - IEEE 5th International Congress on Information Science and Technology, pp. 172–177, Marrakech, Marocco, 21-27 October 2018
DOI: 10.1109/cist.2018.8596505
Metrics:
Savino P., Tonazzini A., Debole F., Salerno E.
Digitalization of ancient manuscripts is becoming a standard in libraries and archives. In many cases, manuscripts suffer of degradations that may require performing different types of elaborations on the digital images, in order to improve their legibility and analyze their contents. Digital archives containing digital images of manuscripts and all the elaborations performed on these images are thus of primary importance for a complete exploitation of all available information regarding the manuscripts themselves. This paper presents a metadata schema suitable for the management of such an archive. The archive will offer the possibility of describing, storing and accessing all the available manuscript versions, and to search them based on their content.Source: CiST 2018 - IEEE 5th International Congress on Information Science and Technology, pp. 172–177, Marrakech, Marocco, 21-27 October 2018
DOI: 10.1109/cist.2018.8596505
Metrics:
See at:
ISTI Repository | doi.org | ieeexplore.ieee.org | CNR ExploRA
2018
Conference article
Open Access
A first step towards NLP from digitized manuscripts: virtual restoration
Debole F., Hanif M., Salerno E., Savino P., Tonazzini A.
Digitization of the documental heritage conserved in libraries and archives is a common practice, in order to ensure the preservation and fruition of this extended part of the human cultural and historical patrimony. For the most precious, fragile and difficult to read and decipher manuscripts, specialized though portable digitization equipment, such as high resolution multispectral/hyperspectral cameras, is nowadays available. Digitization made it possible the increasingly extensive use of digital image processing techniques, to perform a number of virtual restoration tasks, which constitute a first, often necessary step prior subsequent automatic analysis of the writing contents, with the ultimate goal to perform automatic transcription and/or natural language processing tasks. Here we report our experience in this field, referring, as a case study, to the problem of removing one of the most frequent and impairing degradation affecting many ancient manuscripts, i.e., the bleed-through distortion. In this case, virtual restoration gives also the immediate benefit to facilitate the work of philologists and paleographers interested in examining and transcribing the manuscript in a traditional way.Source: CiST 2018 - IEEE 5th International Congress on Information Science and Technology, pp. 188–193, Marrakech, Marocco, 21-27 October 2018
DOI: 10.1109/cist.2018.8596494
Metrics:
Debole F., Hanif M., Salerno E., Savino P., Tonazzini A.
Digitization of the documental heritage conserved in libraries and archives is a common practice, in order to ensure the preservation and fruition of this extended part of the human cultural and historical patrimony. For the most precious, fragile and difficult to read and decipher manuscripts, specialized though portable digitization equipment, such as high resolution multispectral/hyperspectral cameras, is nowadays available. Digitization made it possible the increasingly extensive use of digital image processing techniques, to perform a number of virtual restoration tasks, which constitute a first, often necessary step prior subsequent automatic analysis of the writing contents, with the ultimate goal to perform automatic transcription and/or natural language processing tasks. Here we report our experience in this field, referring, as a case study, to the problem of removing one of the most frequent and impairing degradation affecting many ancient manuscripts, i.e., the bleed-through distortion. In this case, virtual restoration gives also the immediate benefit to facilitate the work of philologists and paleographers interested in examining and transcribing the manuscript in a traditional way.Source: CiST 2018 - IEEE 5th International Congress on Information Science and Technology, pp. 188–193, Marrakech, Marocco, 21-27 October 2018
DOI: 10.1109/cist.2018.8596494
Metrics:
See at:
ISTI Repository | doi.org | CNR ExploRA
2017
Journal article
Open Access
Removal of achromatic reflections from a single color image
Bedini L., Savino S., Tonazzini A.
In this paper we consider the problem of removing achromatic reflections from a picture of a scene taken through a semi-transparent medium, assuming that the reflection pattern is due to a light source or another object located in front of the object of interest. While other works assume the availability of multiple observations, we consider the more challenging problem of having as data a single color image. We suppose a data model where the virtual reflected image combines additively with the real transmitted image of the object, through unknown coefficients. This highly underdetermined problem is handled by means of a blind estimation technique that exploits the strict dependence of the gradients of the three color channels of the ideal image, and their independence from the gradient of the grayscale reflected image. The model parameters are estimated through independent component analysis, and then the component images are estimated through a regularization technique. The whole algorithm is very fast, and its performance is quantitatively evaluated on numerically generated images, and qualitatively tested on real images.Source: Pattern recognition and image analysis 27 (2017): 675–685. doi:10.1134/S1054661817040034
DOI: 10.1134/s1054661817040034
Metrics:
Bedini L., Savino S., Tonazzini A.
In this paper we consider the problem of removing achromatic reflections from a picture of a scene taken through a semi-transparent medium, assuming that the reflection pattern is due to a light source or another object located in front of the object of interest. While other works assume the availability of multiple observations, we consider the more challenging problem of having as data a single color image. We suppose a data model where the virtual reflected image combines additively with the real transmitted image of the object, through unknown coefficients. This highly underdetermined problem is handled by means of a blind estimation technique that exploits the strict dependence of the gradients of the three color channels of the ideal image, and their independence from the gradient of the grayscale reflected image. The model parameters are estimated through independent component analysis, and then the component images are estimated through a regularization technique. The whole algorithm is very fast, and its performance is quantitatively evaluated on numerically generated images, and qualitatively tested on real images.Source: Pattern recognition and image analysis 27 (2017): 675–685. doi:10.1134/S1054661817040034
DOI: 10.1134/s1054661817040034
Metrics:
See at:
ISTI Repository | Pattern Recognition and Image Analysis | link.springer.com | CNR ExploRA